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Abstract —Hardware-based Trusted Execution Environments 
(TEEs) are widely deployed in mobile devices. Yet tbeir use 
has been limited primarily to applications developed by the 
device vendors. Recent standardization of TEE Interfaces by 
GlobalPlatform (GP) promises to partially address this problem 
by enabling GP-compliant trusted applications to run on TEEs 
from different vendors. Nevertheless ordinary developers wishing 
to develop trusted applications face significant challenges. Access 
to hardware TEE Interfaces are difficult to obtain without support 
from vendors. Tools and software needed to develop and debug 
trusted applications may be expensive or non-existent. 

In this paper, we describe Open-TEE, a virtual, hardware- 
independent TEE implemented in software. Open-TEE conforms 
to GP specifications. It allows developers to develop and debug 
trusted applications with the same tools they use for developing 
software in general. Once a trusted application is fully debugged, 
it can be compiled for any actual hardware TEE. Through 
performance measurements and a user study we demonstrate 
that Open-TEE is efficient and easy to use. We have made Open- 
TEE freely available as open sourc^ 


I. Introduction 

Personal computing devices such as smartphones, tablets 
and laptops have become pervasive. They are used to store 
sensitive data and access critical services across a wide range 
of domains, such as banking, health care and safety, where 
privacy and security are paramount. On the other hand, tra¬ 
ditional operating systems and the services that they provide 
are becoming so large and complex that the task of securing 
them is increasingly harder. Hardware-based trusted execution 
environments (TEEs) were developed to address this gap. A 
TEE on a device is isolated from its main operating environ¬ 
ment by using hardware security features. It offers a smaller 
operating environment that provides just enough functionality 
so that sensitive data and operations can be offloaded to it. 

Hardware-based TEEs have been widely deployed in mo¬ 
bile devices for over a decade |[T|. TI M-Shield ^ and ARM 
TrustZone 0, HI are early examples, followed by newer 
architectures like the Intel SEP security co-processor 0 and 
Apple’s “Secure Enclave” co-processor 0. Business require¬ 
ments such as the need to enforce digital rights management 
and subsidy locks, as well as regulatory requirements like 

* https://open-tee.github.io 


cloning- and theft protection have been the driving forces 
behind such large scale deployment HI. Such requirements 
continue to appear; e.g., fingerprint scanners with hardware 
protection, hardware-backed keystores, and the recent “kill 
switch’ ’ m bill in California mandating that a mobile device 
must be capable of being rendered inoperable if it is stolen. 

Although the early hardware security modules (HSMs) like 
the IBM cryptocard^ were programmable the vast major¬ 
ity of HSMs used with personal computers and servers today 
are typically application-specific modules or fixed function co¬ 
processors like the Trusted Platform Modules (TPMs) M- In 
contrast, TEEs in mobile devices are programmable. However, 
despite widespread deployment of hardware-based TEEs in 
mobile devices, application developers have lacked the inter¬ 
faces to use TEE functionality to protect their applications 
and services. Nor have they been researched extensively in the 
academic community. Recent efforts by GlobalPlatform cni 
to specify standard interfaces for TEE functionality in mobile 
devices HD will partially address this problem. However, there 
are a number of factors that stand in the way of widespread 
use of hardware-based TEEs in application development and 
research. Chief among them is the difficulty of developing 
applications for TEEs. Software development kits for TEE 
application development are often proprietary or expensive. 
Debugging low-level TEE applications either requires expen¬ 
sive hardware debugging tools, or leaves the developer with 
only primitive debugging techniques like “print tracing” (e.g., 
using printf statements in C to keep track of how values of 
variables change during program execution). 

In this paper, we argue that a virtual standards-compliant 
TEE implemented entirely in software will allow developers 
to build TEE applications using tools and development envi¬ 
ronments that they are already familiar with. It will also allow 
applications to be tested and refined even when developers do 
not have access to devices where hardware TEE functionality 
has been made accessible to them. Such a facility will greatly 
ease TEE application development and can trigger new ways 
of using TEEs. We make the following contributions: 

• We design and implement such a virtual TEE called 
Open-TEE which conforms to GlobalPlatform Spec¬ 
ifications. We identify requirements that would make 
Open-TEE acceptable to developers and make specific 

^ http://www.ibm.com/security/cryptocards/ 
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Fig. 1; A TEE in a Computing Device 
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III I. Open-TEE is publicly available on GitHub0 


We show that Open-TEE is efficient, hardware- 
independent and allows a developer to carry out much 
of the development life cycle of standard-compliant 
TEE applications using popular application develop¬ 
ment environments they already use. We demonstrate 
that Open-TEE significantly improves the ease-of- 
use of TEE application development by conducting a 
small-scale, yet rigorous, user study with experienced 
professional TEE developers (Section [TV]). 


Given the demonstrable usability benefits, we recommend that 
organizations who develop applications for TEEs should con¬ 
sider incorporating Open-TEE into their development process. 
We also hope that this paper will enable more researchers to 
discover the power of TEEs and use Open-TEE to develop and 
experiment with new TEE applications. 

Note that Open-TEE is not intended to emulate a particular 
hardware TEE. The goal of Open-TEE is that a Trusted Appli¬ 
cation developed successfully with Open-TEE is guaranteed to 
compile and run on any target GlobalPlatform-compliant TEE. 


II. Background 
A. Structure of a TEE 

A TEE is a secure, integrity-protected processing envi¬ 
ronment, consisting of processing, memory and storage ca¬ 
pabilities. Eigure [T] shows how a device can be visualized 
as a series of distinct environments with their own set of 
features and services. Using the terminology introduced by 
GlobalPlatform ifT^ we describe below the concepts illustrated 
in Eigure 

Rich Execution Environment (REE): The word “rich” here 
refers to an operating environment that is feature rich such as 
one would expect from a modern platforms such as Android, 
iOS, Windows, Linux or OS X. 

Trusted Execution Environment (TEE): The TEE is a com¬ 
bination of features, both software and hardware, that isolate 
the execution of tasks from the REE. These environments have 
a limited set of features and services as they are intended to 

‘ http ://open- tee. github.io/ 


only address the security critical subset of an application’s 
functionality such as offloading some cryptographic operations 
or key management. 

Trusted Application (TA): An application encapsulating the 
security-critical functionality to be run within the TEE. This 
may be a service style application that provides a general 
feature, such as a generic cryptographic keystore, or it could 
be designed to offload a very specific part of an application 
that is running in the REE, such as a portion of the client state 
machine in a security protocol like TLS. 

Client Application (CA): CAs are ordinary applications (e.g., 
browser or e-mail client) running in the REE. CAs are respon¬ 
sible for providing the majority of an application’s functional¬ 
ity but can invoke TAs to offload sensitive operations. 

As an example, consider a common use case for TEEs: 
the offloading of Digital Rights Management (DRM) protected 
content. The CA would be responsible for the majority of 
the tasks associated with viewing the content i.e. opening the 
media file, providing a region in the display into which it 
can be rendered (the window) and providing a mechanism to 
start, stop, rewind the media. A TA would be used to decrypt 
the protected media stream and make the decrypted content 
available directly to the graphics hardware that is responsible 
for rendering and displaying the stream. 

B. TEE architectural options 

A TEE can be realized in different ways, but the overall 
concept stays the same. Eigure shows a number of ways in 
which these TEEs can be realized: 

Co-Processor: A separate core, generally with its own periph¬ 
erals, is used to offload the security critical tasks from the main 
operating environment. The benefits of such a configuration are 
that the operation can generally be completely isolated and it 
can run simultaneously with the main core. The drawback is 
that there is an overhead associated with transferring the data 
to and from the core. Also, the co-processor is generally less 
powerful than the main core. The co-processor design can be 
further separated into two alternatives: 

External Security co-processor: is a discrete hardware module 
outside the physical chip (commonly referred to as “System on 
Chip” or SoC) containing the main core, and is thus completely 
isolated from it, not sharing any resources with it. 

Embedded Security co-processor: is embedded into the main 
SoC and thus has the capability to share some of the resources 
of the main system. It is still isolated from the main processor. 

Processor Secure Environment: Many popular mobile TEE 
architectures follow a configuration where a single core sup¬ 
ports multiple virtual cores that are mutually exclusive of one 
another i.e. when one is running the other is suspended. Gen¬ 
erally there is some form of trigger to allow the core to switch 
from one state to the other. This configuration is sometimes 
referred to as the “processor secure environment’ ’ IH. 

ARM TrustZone is an example of this configuration. In Trust- 
Zone, the processor core can be in one of two “worlds”: a 
“secure world” (for the TEE) and a “normal world” (for the 
REE). A special instruction called Secure Monitor Call (SMC) 
can be executed to trigger the processor running in normal 
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Fig. 2: Three potential architectural options for realizing a TEE architecture (adapted from IfT^ l 


world to enter “monitor mode” that marshals the transition to 
secure world la. The advantage of this conhguration is that 
there is no need to offload the data to and from the secure 
world. However, there is a cost associated with having to store 
and restore the device state on entry and exit from a given 
mode. On single core devices there is also an added security 
benefit from having only one world running at a given time 
in that it ensures that the normal world OS cannot interfere 
with the secure world directly or indirectly (e.g., software side- 
channel attacks). However, this also has the disadvantage that 
when one world is active the other world must be completely 
halted, thus complicating interrupt handling. 

Intel Software Guard Extensions (SGX) m, ca is another 
example of such a variant, the core does not perform a full 
transition to and from a secure world. Instead parts of a 
standard application, both code and data, are protected by 
mechanisms in the core. Parts of the application, called an 
Enclave, are encrypted by a key that is only accessible to the 
CPU. When an “enter enclave (EENTER)” lIT^ instruction is 
received the code and data are decrypted and operated upon 
in the core. They never leave the CPU package unencrypted, 
thus protecting them against external access. The benehts are 
that there is no need to transfer data back and forth between 
cores or to setup complicated transitions to and from a secure 
world, and there is no additional need for a separate operating 
environment as is required in other styles of TEE conhguration. 

Virtualization based on hardware features such as AMD-V 
and Intel VT-(x,d), have existed for many years and are used 
extensively to provide separation of resources between differ¬ 


ent operating environments especially in high density server 
conhgurations. They rely on processor support to allow virtu¬ 
alization of instructions and access to resources e.g. through 
the use of an I0MML|^ access to and from peripheral devices 
can be restricted. Though in and of themselves they are not 
designed solely to provide a TEE, there is recent research ini, 
m to see how these can be used as an alternative to dedicated 
hardware based TEEs. When deployed as TEE environments 
they generally rely on a Virtual Machine Manager (VMM) to 
provide the marshaling of access to the resources. This poses 
a security concern as the VMM is considered to be part of 
the Trusted Computing Base (TCB) of such a system, thus 
increasing the attack surface. 

C. Standardizing TEE Functionality 

The landscape for TEEs has been very diverse, with a 
variety of different architectural options from multiple man¬ 
ufacturers. Even platforms using the same type of TEE are 
often not inter-operable. Eor example, an application written 
for one TrustZone-based platform will generally not run on 
a different TrustZone-based platform. They may be using a 
different TEE OSs or different REE OS drivers. On the other 
hand, developers and others who are higher up in the software 
ecosystem are less concerned with intricacies of low-level 
software or hardware but more concerned with their ability 
to use the capabilities of TEEs easily and across different 
platforms. This calls for standardization. 
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One initiative in TEE standardization has been undertaken 
by GlobalPlatform (GP) d, which “is a cross industry, 
non-profit association which identifies, develops and publishes 
specifications that promote the secure and inter-operable de¬ 
ployment and management of multiple applications on secure 
chip technology” lfT9ll . GP offers specifications in three areas; 
smartcards, back-end support systems and devices. In this 
paper, we are concerned with specifications from the device 
working group related to the APIs for TEEs. 

Eigure shows the primary interfaces standardized by GP. 
The TEE Core API provides an extensive set of features such as 
a crypto API and secure storage that can be used to implement 
a TA, for example a DRM decoder. The TEE Client API is 
a very generic and thin layer consisting of a small number 
of functions and definitions that allow the transfer of data 
back and forth from the REE to a TA. A CA, for example 
a DRM player, will implement all complex but non-critical 
functionality by itself, but use the Client API to invoke the 
corresponding TA, such as the DRM decoder. Between the 
TEE Client API running on the REE and TEE Core API 
running on the TEE we have an effective Remote Procedure 
Call (RPC) mechanism where a process running in the REE 
can invoke tasks in the TEE. 

These standardization efforts in GlobalPlatform could re¬ 
solve the issue of inter-operable TEEs. However, they do not 
remove the obstacle in gaining access to the requisite hardware 
nor does simplify the task of developing and testing TAs. 

III. Open-TEE 

In order to pave the way for the widespread use of TEE 
functionality by developers and researchers we propose an 
architecture and a software development kit (SDK) that imple¬ 
ments this as a framework atop a set of tools that are familiar to 
the developer, thus removing the need for specialized hardware 
and the overheads that it incurs. 

A. Motivation 

To explain our motivations, we now revisit the difficulties 
in developing TEE applications that we alluded to in Section |I] 


a) Enable developer access to TEE functionality: Eor 
a variety of reasons, access to TEEs is generally restricted to 
developers working for chip manufacturers and to the original 
equipment manufacturers (OEMs) that make devices based on 
these chips. Usually, the technology is proprietary and easily 
deployable SDKs are not available. Eurthermore, TEEs may 
not have a security architecture within them to safely allow 
complete outsider developers access to TEEs. However, there 
have been attempts to address this problem lESI. 

b) Provide a fast and efficient prototyping environment: 
The most common methods of debugging TAs are to either 
use expensive JTAG01ebugging or resort to primitive “print 
tracing” by inserting diagnostic output in the source code. 
The former generally allows for detailed instruction level 
debugging. However, the costs associated with these debuggers 
can be prohibitively expensive, and the setup complex. Print 
tracing as a debugging technique is cumbersome and clutters 
up the source code even to locate the source of a problem. 
Another concern encountered by TEE developers is that if a 
TA running on actual device hardware crashes, a hard reset 
of the device maybe required to recover, thereby significantly 
increasing the time and effort of debugging. 

c) Promote research into TEE Services: Ways to isolate 
TEEs from REEs are reasonably well understood as we saw 
in Section What is less well understood are the types 
of services that could benefit from using TEEs. As the app 
store model has proven, given an opportunity, the developer 
community at large is capable of pushing the boundaries and 
exploring new and novel ways to use technology. Making it 
possible for researchers to easily develop TAs could trigger 
the development of novel and innovative applications. 

d) Promote community involvement: The pre-requisite 
for involving the developer community and researchers at 
large is to allow them access to a freely and easily available 
development environment, SDK and a platform with which 
to experiment. The financial and technical aspects of making 
hardware TEEs available for development on a large scale mo¬ 
tivates the need for a software framework for TA development 
which is not bound to any particular hardware or vendor. 

B. Requirements 

Motivated by the above discussion, our aim is to develop 
an SDK and framework that allows for the development and 
testing of standard-compliant TEE applications. The frame¬ 
work should allow development of GP-compliant CA and 
TA functionality without having to rely on any particular 
hardware support. We intend this to be a fast prototyping 
and development environment that also provides a platform 
from which to conduct further research into TEE functionality. 
Our fundamental design principle is that it should require as 
little configuration and maintenance as possible, allowing the 
developer to focus on the task at hand. 

We identify the following criteria by which we can mea¬ 
sure our TEE framework’s usefulness and hence its potential 
success in addressing the issues that motivated it; 

Compliance; Our framework should comply with GP’s main 
interfaces, the Client and Core APIs. 

^Joint Test Action Group standard addresses debugging of integrated circuits 




































Hardware-independence: As a software based solution our 
framework should not be dependent on a particular TEE 
hardware environment. It should also not be dependent on any 
particular hardware for the development system itself. 

Reasonable Performance: To be readily deployed, our frame¬ 
work must not suffer from code bloat that adds to the on-disk 
footprint nor to the memory consumption required to run it. 
In addition the start-up and restart times of the environment, 
especially that of the CAs and TAs should not be excessive. 

Ease-of-use: The solution should be easily deployed and 
conhgured. It should use tools that are widely available making 
it more attractive (e.g., there should be no need for extra 
package/tool configuration on the development system). 

We now describe our design and implementation of such 
a software framework which we call Open-TEE. 

C. Architecture 

We begin with an overview of the structure of the Open- 
TEE environment. Eigure identifies the main components 
and their relationships. The color code used in Eigure]^ is the 
same as that used for Eigure[^to make the correspondence be¬ 
tween the Open-TEE implementation architecture and the GP 
conceptual architecture is clear. We describe each component 
in detail below. 




a) Base: Open-TEE is designed to function as a dae¬ 
mon process in user space. It starts executing Base, a process 
that encapsulates the TEE functionality as a whole. Base is 
responsible for loading the configuration and preparing the 
common parts of the system. Once initialized the Base will 
fork to create two independent but related processes. One 


process becomes Manager and the other. Launcher which 
serves as a prototype for TAs. 

b) Manager: Manager can be visualized as Open-TEE’s 
“operating system.” Its main responsibilities are: managing 
connections between applications, monitoring TA state, pro¬ 
viding secure storage for a TA and controlling shared memory 
regions for the connected applications. Centralizing this func¬ 
tionality into a control process can also be seen as a wrapper 
abstracting the running environment (e.g. GNU/Linux) and 
reconciling it with the requirements imposed by the GP TEE 
standards. GP requirements and the host environment’s func¬ 
tionality are not always aligned. Eor example, GP requirements 
stipulate that if a TA/CA process crashes unexpectedly, all 
shared resources of the connected processes must be released. 
In a typical running environment, this requires additional steps 
beyond just terminating the process. Eor example all shared 
memory must be unregistered - this needs to be a distinct 
action from normal process termination. 

c) Launcher: The sole purpose of Launcher is to create 
new TA processes efficiently. When it is first created. Launcher 
will load a shared library implementing the TEE Core API 
and will wait for further commands from Manager. Manager 
will signal Launcher when there is a need to launch a new 
TA (for example, when there is a request from a CA). Upon 
receiving the signal. Launcher will clone itself. The clone will 
then load the shared library corresponding to the requested TA. 
The design of Launcher follows the “zygote” design pattern 
(such as that used in Android HT]) of pre-loading common 
components. This is intended to improve the perceived per¬ 
formance of starting a new TA in Open-TEE because shared 
libraries and configurations common to all TAs are pre-loaded 
into Launcher, the time required to start and conhgure the 
new process is minimal. A newly created TA process is then 
re-parented onto Manager so that it is possible for it to control 
the TA (so that, for example, it can enforce the type of GP 
requirements discussed in the paragraph above). 

d) Trusted Application Processes: The architecture of 
the TA processes is inspired by the multi-process architecture 
utilized in the Chromium Project ED. Each process has 
been divided into two thread^ The first handles Inter-Process 
Communication (IPC) and the second is the working thread, 
referred to respectively as the lO and TA Logic threads. 
This architectural model enables the process to be interrupted 
without halting it, as occurs when changing status flags and 
adding new tasks to the task queue. Additional benehts of this 
model are that it allows greater separation and abstraction of 
the TA functionality from the Open-TEE framework. 

e) GP TEE APIs: The TEE Client API and TEE Core 
API are implemented as shared libraries in order to reduce 
code and memory consumption. 

f) IPC: Open-TEE implements a communication pro¬ 
tocol on top of Unix domain sockets and inter-process signals 
as the means to both control the system and transfer the 
messages between the CA and TA. 

D. Implementation and Tooling 

a) Utilizing existing functionality: To meet the 
hardware-independence requirement, we do not emulate spe- 

®The architecture of Manager follows the same division 































cific TEE hardware with software based emulators, such as 
QEMU E3i . Instead we rely on existing technologies and 
the services offered by the mainstream OS of Open-TEE’s 
running environment rather than developing a new TEE OS 
to deploy the GP APIs in. In addition we reuse software from 
existing open source projects, such as the OpenSSL crypto 
library and the GNU tool suite, thereby reducing the amount 
of time required to develop and test the Open-TEE framework. 

This also contributes towards meeting the ease of use 
requirement in that developers can easily set up Open-TEE and 
start developing TAs using a set of familiar tools, editors, IDEs, 
compilers and debuggers. Eor example, a developer utilizing 
Open-TEE can connect to a TA process with a cheap reliable 
software debugger such as GDB ll24l for detailed debugging 
tasks like stepping through the code, inspecting variables and 
registers etc. 

b) Development process: The intended user base for 
Open-TEE consists of seasoned developers. To ensure viabil¬ 
ity in such a demanding user base, we adopted a rigorous 
development process for Open-TEE so that the end result will 
be perceived as robust and usable. Open-TEE is developed 
as an open source project and as such there are a number 
of powerful tools that are freely available for this type of 
project. As previou^ mentioned GitHub is used for hosting 
the code. GerritHutnis used for performing peer-review of all 
code before it is submitted to the code base. In addition to the 
manual review process we leverage the power of Cover! t}J^ to 
perform in depth static analysis scans. This enforces secure 
coding practices and helps to find potential functional bugs 
that may have been missed during the manual code review. In 
addition, we have deployed a continuous integration (Cl) server 
running Jenkin^ which we have connected to GerritHub. Its 
main task is to perform a number of “smoke tests’[^ on the 
new patches. These tests ensure that the patches conform to the 
coding guidelines, build successfully and that the basic system 
is usable after the patches are applied. 

c) Open-TEE in Use: Being designed as an open source 
framework upon which to build and test features that will 
utilize a TEE, Open-TEE has been implemented to be as incon¬ 
spicuous as possible. The complexity of the system is hidden 
from the users of Open-TEE. They are presented with an SDK 
that exposes the Client and Core APIs without being required 
to have a deep understanding of how the overall framework 
works, thereby allowing them to focus on the development of 
their own TAs. However, Open-TEE is already being extended 
by the community. The ongoing implementation of the GP 
Trusted UI specification is an example. 


IV. Evaluation 

We now return to the requirements from Section Inland 
evaluate how well Open-TEE meets those requirements. 


^ http ://gerrithub. io/ 

®https://scan.coverity.com/projects/3441 
®http://jenkins-ci.org/ 

’®A suite of tests intended to ensure that the basic functionality of a system 
are intact. 


A. Compliance 

Every effort has been made to comply with the GP 
standard. Whenever this has not been feasible, due to time 
constraints or in the interest of providing a platform upon 
which to build, the deviation has been documented and a 
debug message is logged to inform the user of the non- 
compliance. The Client API is fully implemented. The Core 
API implementation has 100% function coverage, however, 
the cryptographic algorithm coverage is currently 80% due to 
the use of existing libraries that do not support the remaining 
algorithms. A compliance test suite is commercially available 
from GP. There is no information about how well existing 
TEE hardware conform to the GP specifications. Based on our 
experience in implementing version 1.0.26 of the GP Core 
API, we provided detailed feedback, including on errors and 
ambiguities in the specification, to GlobalPlatform in response 
to their solicitation of public comments. Several items in our 
feedback have been addressed in the released version 1.1. 


B. Hardware-independence 

By following the GP standard and not emulating any 
specific TEE hardware, Open-TEE is independent of TEE 
hardware. TAs developed with Open-TEE can be compiled to 
any target TEE hardware architecture. We have verified ll25l 
that a non-trivial a TA developed using Open-TEE (284 LoC, 
19 Internal Core API invocations (9 unique functions), 6 
invokable TA commands) has been succesfully compiled and 
run on a hardware TEE based on ARM TrustZone running the 
Trustonic <t-base environment E6\ . 

Open-TEE can provide coverage reports to help highlight 
hot-spots in the code, generate call graphs etc. The GP Internal 
Core API includes memory management primitives and allows 
configuration parameters (such as gpd. ta . dataSize and 
gpd. ta . stackSize) to indicate how much heap and stack 
memory is available to a TA. A developer can use these 
parameters to configure Open-TEE to reflect the memory 
restrictions of a target hardware TEE environment. However, as 
the actual TEE is potentially running a different environment 
than that offered by Open-TEE - possibly utilizing hardware 
based cryptographic accelerators, potentially having a different 
CPU, with different clock speed and throughput characteristics 
- it will result in different timing characteristics. In this sense, 
as with all virtual environments, Open-TEE can not fully 
replace the actual hardware environment for the final stages 
of the development cycle. Instead developers using Open-TEE 
can gain confidence that the hardware-independent parts of 
their trusted applications have been optimally implemented by 
making judicious use of coverage reports and other generic 
analysis techniques. Any hardware-specific optimization, such 
as performance tuning, naturally needs to be done on the target 
hardware environment. 

Open-TEE has been deployed and used on various devel¬ 
opment environments ranging from servers to desktops and 
laptops. We have tested Open-TEE on both ARM and x86 
architectures. Open-TEE requires Linux but has been run 
successfully on virtual machines hosted on other OSs. 



TABLE I; Binary sizes (bytes) 



Text 

Data 

BSS 

overall 

libInternalApi.SO 

117448 

2248 

160 

119856 

libtee.so 

18617 

880 

152 

19649 

Total Framework 

224948 

7760 

1664 

234372 


TABLE II: Memory usage (KB) 



RSS 

Shared 

Private 

PSS 

Manager 

1024 

764 

260 

305 

Launcher 

1624 

1232 

392 

558 

Manager 

1112 

832 

280 

316 

Launcher 

1648 

1548 

100 

397 

Test TAj^ 

1072 

932 

140 

308 

Manager 

1116 

832 

284 

319 

Launcher 

1648 

1548 

100 

337 

Test TAl 

1072 

944 

128 

245 

Test TA^ 

1236 

1068 

168 

299 


C. Footprints and Performance 

To evaluate our performance we deployed Open-TEE on a 
desktop machine (Intel i7-2600 CPU with 8GB RAM) running 
64-bit Ubuntu 14.04. All performance tests were run 40 times 
while the machine was under normal load e.g. having editors 
and browsers open. 

1) Disk and Memory consumption: Open-TEE is written 
in ANSI C with a total of 12423 line^ spread over 78 source 
and header hies. Table U shows the total size of the framework 
and highlights two libraries from the framework that are of 
most interest to developers, being libInternalApi . so 
against which the TAs are linked and libtee.so against 
which CAs are linked. As is standard on operating systems 
that support shared libraries the “Text” section, containing the 
programs code, can be shared among the different processes 
that link against it. The “Data” and “BSS” respectively refer 
to the initialized and uninitialized data parts of the library that 
can be shared in a Copy-On-Write (COW) basis. As the table 
highlights the vast majority of the libraries’ size can be shared, 
thus reducing the required footprint. 

Table [B shows the memory consumption of the running 
process under three different scenarios. The hrst shows the 
memory consumption of Manager and Launcher immediately 
after they have been launched, i.e. before any TAs have 
been launched. The next section shows how the memory 
consumption increases when one TA is launched while the 
last sections shows the situation when two TAs are running 
simultaneously. 

RSS (Resident Set Size) shows how much memory has 
been allocated for a process, this includes all memory that 
a process shares with other processes. As such it is very naive 
measurement of a processes memory impact. 

Shared is the memory that a process shares with other 
processes, i.e. through the use of shared libraries. 

Private is the memory that is private to a process and will be 
returned to the system when the process terminates, however, 

"gathered using sloccount: http://www.dwheeler.com/sloccount/ 
"ta_conn_test_app ~100 lines of C 
"example_digest_ta -140 lines of C 


TABLE III: Average build and execute times of a TA, including 
standard deviations 


Time 

Build 

147 ms ± 10.95 

Execute 

430.5 MS ± 32.6 


Copy-On-Write semantics after a process fork may complicate 
this calculation. The Private pages may actually be shared until 
one or the other of the processes tries to write to the page, at 
which time it will be given its own copy of the Private page. 

PSS (Proportional Set Size) is a realistic indicator of the 
actual memory footprint of a process. It is calculated as the 
sum of the Private memory used by a process and the average 
Shared memory use per process. E.g., if a process has 100KB 
of Private memory and 1000KB of memory shared with 10 
processes, its impact on system memory is 200KEp] Taking 
the example of Launcher between runs 2 and 3 we see that 
while the RSS, Shared and Private memory usage stay constant 
the PSS decreases as more pages are shared with the new TA. 

Overall, we can conclude that (a) the memory footprint of 
Open-TEE is low and (b) the extensive use of shared libraries 
implies that the marginal memory cost of launching a new TA 
is small, as shown by the PSS hgures. 

2) Build and Run performance: One of the driving re¬ 
quirements of Open-TEE is the need to have short build and 
deploy cycles to help reduce the overall development effort. 
Table [HI] highlights that Open-TEE does not pose a signihcant 
overhead to the developer, taking an average of just 147 ms to 
perform an incremental build of a TA. The time required for 
an incremental build was comparable to that of a clean build, 
falling within the standard deviation of the former, this can be 
attributed to the source code being conhned to a single C file. 
Comparative results are not available for deployed hardware- 
based development environments. However, considering that a 
full reset of the target device and the subsequent boot of its OS 
may be required before the CA can be launched, Open-TEE’s 
performance is likely to be perceived as being superior. 

D. Ease of use 

Determining whether Open-TEE eases the burden of TA 
development is particularly challenging because, until now, TA 
development has been limited to a very small set of developers. 
Eortunately, we were able to recruit several experienced TA 
developers from multiple organizations to participate in a user 
study. Our user study was conducted as follows. 

Participants: Eourteen people participated in the study. All 
had prior software experience (between 3 and 33 years, 
M = 13, SD = 8.2). Eleven had prior experience devel¬ 
oping/debugging TAs (between | and 15 years, M = 5.1, 
SD = 4.2). 

Materials: The Standard System Usability Scale (SUS) ll27l . 
1281 questionnaire was used to elicit the participants’ estimates 
of the ease of use in developing TAs. We used a pre-study 
and a post-study questionnaire. In addition to demographic 
information, the pre-study questionnaire included free-form 

"lOOKB + (1000KB / 10) = 200KB 
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SUS score by participant 


Software and TA development experience by participant 



Particf)ar( 


Fig. 5; Pre- and post-study SUS score (for participants with 
prior TA development experience) 



Paitcipant 


Fig. 6; General software and TA development experience 
by participant (for participants with prior TA development 
experience) 


TABLE IV; Complexity metrics for TA used in user study 


Total Physical Source Lines of Code 284 

Total Number of Invokable TA Commands 6 

Total Number of Internal Core API Calls 16 

Number of Distinct Internal Core API Calls 9 


questions about the current software development environment 
(if any) they use for TA development. It also contained a SUS 
questionnaire which the participants were asked to complete 
with their current TA development environment in mind. This 
was completed only by those participants who had prior TA 
development experience. The material for the user studjj^ask 
was a sample CA/TA pair, provided as part of the Open-TEE 
source tree. A software flaw had been introduced to the TA, 
which, when executed, would result in a segmentation fault 
and subsequent premature termination of the TA. The CA was 
free of error and was only used to interact with the TA running 
in Open-TEE. Code complexity metrics for the TA used in the 


user study are shown in Table IV 


The post-study questionnaire consisted of a SUS form 
which the participants were asked to complete with Open- 
TEE in mind. The questionnaire also had open ended questions 
about specific difficulties they face in TA development. 


Procedure; The user study was conducted in three steps. In 
the first step, participants were first asked to complete the pre¬ 
study questionnaire. After this they were pointed to a web page 
containing brief instructions on how to install and use Open- 
TEE. In the second step, once the participants completed the 
tutorial they were told about the flawed TA. They were tasked 
to identify the reason for the TA malfunction using Open-TEE 
and correct the flaw in the TA. Einally, in the third step, after 
the participants had completed the debugging exercise, they 
were asked to complete the post-study questionnaire. 


Results; The mean, standard deviation and median of the SUS 
scores for all participants, including those without prior TA 
development experience are shown in Table |V] With both sets 
of participants, the post-study questionnaire yields a mean 
score above 68, which is considered the threshold value for 


The material can be found at http://open-tee.github.io/userstudy/ 


TABLE V; Mean, standard deviation and median for the pre- 
and post-study SUS scores 



Mean 

Std.dev. 

Median 

Pre-Study SUS 

51.82 

24.70 

62.50 

Post-study SUS 

74.09 

15.01 

77.50 

Post-study SUS (all participants) 

69.92 

18.09 

68.75 


an above average SUS score, indicating an acceptable level of 
usability in Open-TEE. 

Eigurej^shows the scores reported both before and after the 
use of Open-TEE by participants with prior TA development 
experience. Nine out of the eleven participants (82%) rated 
Open-TEE higher than the development environment they are 
using currently. This suggests that the perceived usability of 
Open-TEE is higher than that of the current tools used by the 
experienced TA developers. In five cases (46 %), the difference 
in SUS scores was 35 or more. In the remaining six cases, 
the difference in SUS scores was 10 or less. A Wilcoxon 
signed-rank test showed that the difference in SUS scores is 
statistically significant (z = —2.50, p < .05, r = —0.53). 

The difference in SUS scores divides the participants 
into two distinct groups. The five participants for whom the 
difference was 35 or more had SUS scores below 60 in 
their pre-study questionnaire. The remaining six for whom the 
difference was 10 or less had pre-study SUS scores over 60. A 
natural question is whether we can discern any other difference 
between the two groups that might explain the difference in 
SUS scores. One possible explanation was that experienced 
software developers were comfortable with their current tools 
and hence did not perceive Open-TEE as being easier to use. 
If this explanation is correct then one can hypothesize that 
developers with many years of general software or TA devel¬ 
opment experience will rate their current development tools 
higher than their counterparts with fewer years of experience 
would. However, a Spearman’s rho correlation test indicated no 
significant correlation between the years of general software 
development experience and the SUS score in the pre-study 
questionnaire (r^ = —.042, p > .05), nor between the years 
of TA development experience, and the SUS score in the pre¬ 
study questionnaire (rs = —.204, p > .05). Eigure shows 




























the software development experience (both general and TA) 
reported by each participant whose SUS scores are shown in 
Figure 

A majority of the experienced TA developers (7 out of 
11, 64 %), reported using hardware tools for debugging TAs 
under development. Four (36 %) used Lauterbacfpj hardware 
assisted debug tools. Three (27 %) used other development 
boards such as Arndalj^ Fidcp^ or DS-^^or actual mobile 
devices. Participant responses highlighted different types of 
difficulties in debugging TAs using only hardware: 

• workflow slowdown due to the need to (cross) com¬ 
pile, load and execute TAs on separate hardware 
{“slow execution (flash, download, reboot, run)”, “de¬ 
bugging TA is slow, you need to cross compile and 
push binary into target hardware”), 

• problems due to the hardware itself being under devel¬ 
opment and hence exhibiting flaws, {“TEE itself might 
not work without problems, because some change have 
been made”), and 

• inconvenience caused by the restricted access to pro¬ 
totype hardware “Main difficulty is that you need 
development hardware, which is problematic when 
working outside the office.”). 

Six participants (55 %) reported that their current development 
environment does not support interactive debugging. But even 
the rest, who used tools like Lauterbach tracing, reported that 
they found it easier to resort to print tracing, whenever they 
needed to examine values of TA variables. 

After having used Open-TEE, several participants com¬ 
mented “debugging is easy” or “debugging is fast” in the 
post-study questionnaire. One participant characterized how 
Open-TEE could be integrated into his existing workflow 
before cross compiling to target hardware: “[Open-TEE ] 
complements nicely my previous SDE - first preliminary testing 
with Open-TEE & gdb & OT_LOG (. .), and only after 
that ARM cross compiler & EVP emulation”. The dominant 
suggestion for improvement was a desire to see more extensive 
documentation for Open-TEE. 

Given the sample size, the results should be taken as 
indicative rather than definitive. However, it is reasonable to 
conclude that Open-TEE has the potential to improve the ease 
of use of developing TEE applications. 

V. Related Work 

Ekberg et al. Ill list several reasons for the underutilization 
of TEEs in devices: e.g., lack of standard APIs and easily 
available SDKs and lack of trust between the different stake¬ 
holders, with OEMs being unwilling to open up their security 
environments to third parties. In this section we will review 
a number of initiatives that have been undertaken to address 
some of these issues and compare these efforts to Open-TEE. 

ObC II 20 I was one of the first attempts to address the 
problem of opening the TEE to third party developers by chal¬ 
lenging the prevailing opinion that a credential system must 

http://www.lauterbach.com/ 

1 ' www.arndaleboard.org/ 

http://www.liewenthal.ee/projects/fido/ 

http://ds.arm.com/ 


be centralized and closed. ObC predates many standardization 
efforts and as such defines a proprietary mechanism by which 
to enable the CA/TA communication and synchronization 
while leveraging the TrustZone architecture to enforce the 
security. On the other hand our work aims to promote standards 
adoption in order to proliferate TEE research and deployment. 


Muthu II 29 II analyzes extending QEMU to support Trust- 
Zone, the feasibility of such a solution, and tries to determine 
if it would be beneficial to the developer community. Winter 
et al. II 30 I go one step further and implement a TrustZone 
emulator as an open source project. However, we were not 
able to find the source code. Open-TEE addresses the issue 
of virtualizing the TEE, however, in contrast we are not tied 
to the emulation of a specific TEE implementation. One issue 
with developing an emulator for the TEE is that it still lacks 
an operating system to run. In section II-C we highlight the 
lack of a standardized OS even among the different TrustZone 
implementations. 


To this end there have been a number of efforts to create an 
OS that is suitable to be deployed in TrustZone OH OH OH . 
All of these are open source solutions which are released 
under various licenses (Table |VI| l. In addition to providing an 
operating system for the TEE both OP-TEE OH and T6 OH 
choose to rely on GP as their RPC mechanism between the 
REE and TEE. TLK Oil on the other hand chooses to provide 
a proprietary communication mechanism. 

Sierraware’s Open Virtualization Ga provides a dual- 
licensed OS implementatiorp^ that also supports the GP 
standards. The commercial products (sierraVisor, SierraTEE) 
provide extended functionality and not being GPL there is no 
requirement for any changes to be made publicly available 
by the license holders as is required with their open source 
offering. Open-TEE is licensed under Apache-V2 giving users 
the flexibility of an open source license without the strict 
copyleft requirements. 

Trustonic’s <t-dev developers program ||2H was created 
to support Trustonic partners who have deployed the <t-base 
TEE implementation. This program provides an SDK, tools 
and consulting with the aim of easing the development and 
testing of TEE applications in deployed hardware solutions. 


All of the OS based solutions have to be ported to sup¬ 
port the various HW environments, increasing the effort of 
maintaining the OS and reducing the users available options. 
Many of them also require that the HW be configured in a 
developer mode, without this setting it is generally not possible 
to deploy custom SW to the TEE, for obvious reasons, further 
restricting the developers options. Open-TEE in contrast does 
not have this HW dependency, thus enabling the users to start 
developing with the framework once they have cloned the 
repositories. Based on the references listed above, we conclude 
that no other project Alls the niche of a fast prototyping SDK 
framework that we have described in this paper. 


VI. Discussion and Conclusion 

We have demonstrated that Open-TEE meets our objective 
of an easy-to-use, hardware-independent software framework 
that allows developers to write and debug GP-compliant TEE 

^^proprietary, GPL 





TABLE VI: Comparison of available alternatives to Open-TEE 



Compliance 

HW-independence 

License 

Open-TEE 

yes 

yes 

Apache-V2 

Open Virtualization I34i 

yes 

no 

proprietary, GPL 

OP-TEE |32l 

yes 

no 

BSD-2,BSD-3 

T6 133) 

? 

no 

7 

TLKpil 

no 

no 

MIT,EreeBSD 

TrustZone Emulator |30l 

? 

no 

7 


applications. We made a deliberate decision to open source 
Open-TEE under Apache-V2 license llTSl . The Apache license 
was selected because it is a recognized open source license and 
it provides additional flexibility for those wishing to use the 
framework. All third party components have been carefully 
selected - we have used only components that have been 
properly licensed and do not set any restrictions for future 
use. This has made it possible for people from outside the 
research group to contribute to Open-TEE. Curi'ently a number 
of extensions are being worked on including support for other 
GP APIs and supporting Client API bindings in Java (for 
Android applications). 

Although the sample in our user study is small, participants 
were drawn from several different organizations with track 
records of TA development. This makes us confident that our 
user study results are valid. It is very difficult at this time to 
conduct larger-scale user studies of TA development because 
the community of TA developers is tiny. Recall that expanding 
the size of the TA developer base is the very motivation for 
Open-TEE in the first place. 

We initially intended Open-TEE as a developer tool. How¬ 
ever, an alternative use has become evident in our discussions 
with service providers. Although use of TEEs can improve the 
security and usability of their service, not all their clients may 
have TEE-equipped devices. Yet the service provider would 
like to present a consistent user experience for their entire 
client base. A possible approach for them is to ship their 
application (CA and TA) with Open-TEE and arrange for the 
CA to use Open-TEE if it cannot detect a real hardware TEE 
on the device. This would allow the service provider to have a 
common provisioning mechanism and offer a consistent user 
experience for all their clients. However, once we cast Open- 
TEE as a potential fall-back TEE in this manner, we need to 
address the question of how best to isolate it from the REE 
in the absence of any hardware support. This is part of our 
current work. 

We re-iterate that Open-TEE is not intended to emu¬ 
late any specific TEE hardware. Open-TEE meets its goal 
of guaranteeing that trusted applications developed using it 
will compile and run on any GP-compliant TEE hardware. 
Hardware-specific aspects, such as performance tuning are 
outside the scope of Open-TEE. 

Our hope in writing this paper is to make the research 
community aware of Open-TEE and encourage researchers 
to use it and contribute to its development. We also believe 
that organizations and developers who already develop TA 
applications will benefit from incorporating Open-TEE into 
their development process. 
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